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A bstrar t 

Recently two standards for MANs, FDDI and l)QI)H v have emerged as 
the primary rompei ilorr. for tin* MAN arena. (boat interest exists in build- 
ing higher speed networks which support large numbers of node and greater 
distance and il is not rh*ar what typos of protocols ao* needed for this type 
of en vi moment. Timm is some question as to whether or not those MAN 
standawls can Im* **:<tend«*d to such environments. 

'This paper investigates tin* ext ensibili ty of FDDI to I lie the Gbps range 
and a. long distance environment. It does this first by showing which specifi- 
cation parameters affect performance and providing a. measure for predicting 
utilization of FDDI. A comparison of FDDI at 100Mbps and lGbps is pre- 
sented. Some specific problems with FDDI are addressed and .modifications 
which improve the viability of FDD! in such high speed networks arc investi- 
gated. 1 


*Tliis work was supported by CIT grant It F-K9-002-0I , NASA grant NAG-1-908 and Sun Mi- 

crosystem grant RF 5900-13. 


1 Introduction 


Network data rates are < ommer rial I v ;i\m liable a I. rales in l.he 100Mbps per rhannel 
class. The two m<>s(. pi < iim nen I. of those an* competing MAN standards FI)I)I[2U] 
and DQI)B(QPSX )(- r >] and much research is nincnl.lv ongoing in an effort to bettor 
understand their performance capabilities and limital .ions[8,2]. Research, however, 
is going forward and a national research initiative' is underway to develop Gbps 
networks to be employed as a backbone for a national research nelwork(f)]. 

Many questions still remain as lo the approach which can best suit the require- 
ments of such a network. A national network will likely transport synchronous and 
asychronous traffic, support large numbers of nodes (at least 100 and likely over 
1000), and bespread over very large distances (over 1000 Kilometers). "Hie impact 
of considering these types of parameter ranges can be very negative for token rings 
due to increased token cycle time and for CSMA/Ci) dne to the increased slot times. 
No known research exists to show how KDDI and DQDB are affected. Current na- 
tional networks are very slow packet networks on the order of JiGKbps. As we move 
towards Gbps speeds, the networks will Ik* much more expensive and efficiency will 
become a much more important factor. 

Increased data rates could be accomplished by focusing on the development of 
transmitter/ receiver devices which are capable of functioning at such high rates[7, 
11,16,18], i.e. just build gigbit speed lasers. 'There are of course numerous problems 
associated with such high speed devices other than the transmitter/recei vers them- 
selves, such as how to build computers which can process data at the rate of the 
network and what types of protocols would work best at these rates. For example, 
[9] suggests that it might be necessary to structure packet sizes to be large in order 
to minimize overhead impact. 

Another approach is to examine how current transmitter/ receiver technology 
can be optimized to improve characteristics such as t hroughput and delay. Parallel 
channels show some promise for improved performance (l .T, 12, 1 4] and is the subject 
of some research. Many architectures have been proposed in an attempt Lo design 
more efficient networks. Strategies have included ‘train* protocols [22,21,10], hy- 
brid CSMA/CD protocols^, 15, 13,12/1], slotted and register insertion rings[6], and 
numerous others. 

In this paper I will examine the viability for scaling KDDI, a 100Mbps token 
ring protocol, to the type of environment mentioned above. A number of problems 
will be defined and a suggestion for performance improvement will be given, 'flic 
suggestion for performance enhancement is applicable to token ring networks at any 
speed and distance and provides a framework for improving other types of networks. 
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2 FDD I 


2.1 Basic Token Ring 

A token ring network is distinguished l>y the in.'itiiKT in which transmission rights 
are granted to nodes on the network. The token packet circulates on the ring, 
passing by each node. If a node’s queue is empty, it simply continues circulating 
the token to the next node. However, if the node has a message in its queue to 
transmit, it will effectively remove the token by changing some information in the 
packet. The token itself is not actually removed, but is transmitted in some altered 
form such that subsequent nodes will not see it as a token and will not attempt 
to transmit. The modified token continues circulating and is eventually consumed 
(not retransmitted) when it reaches the no do winch is in the process of transmitting 
(bolding the token). 

Figure I illustrates how the token is ‘removed 1 . Node -1 has a message for node 2 
and is waiting for the token before initialing, transmission. Figure I./; shows that the 
token has been modified so that it will not be recognized as such and the message 
has been placed on the network. Part r shows that I be token has been retransmitted 
on the network and is available for subsequent use. 

As the token continues around the ring, subsequent nodes remove the token and 
append another packet. This scenario is illustrated in Figure 2 where node 1 has a 
message for node In order to send I he* message, I he token is modified, the message 
for node * r ) is transmitted and a. new token is made available to the other nodes. The 
network becomes filled wilh messages and old hdrns. At most one real token is on 
the network at any instant of time*. 

The exact time at which the old tokens and messages are removed is dependent 
upon the distance of the network, the length of the packet and the data rate of 
the network. The most important factor to note is that all nodes except the node 
bolding the token are forwarding messages. The node bolding the token does not 
forward the incoming message but instead forwards its own message. Incoming 
messages arc lost. Eventually, the old token will encounter a node which is in the 
process of transmitting a message and be removed. 

Data packets arc removed in a slightly different manner. It is important that 
they be removed by the sender so that a receiver will not accept the message a 
second time if it recirculates. 'The tail of the message is reinoved(modifted) at the 
sender once the address is recognized, whereas the fragments of the headers of these 
[jackets arc removed as mentioned above[H)]. 


2.2 Token Rotation Time 

The decentralized access mechanism of a token ring [irol.or.ol can place limitations 
on how long a station has access to the network once it obtains the token. The 
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Figure 1: Token I v in Protocol Token Capture 

approach in FI)I)I is f.o limil. the amount of Hwr for which one staion can hold 
the tokeu[2.'i] . Fach node I in s a timer wliicli is reset when the l.okcn arrives. When 
the token returns, the node may rapture it only for an amount of lime which will 
assure that the token will return within a. specific time period. This time period is 
the Tavf/r.l lokr.ii Rotation YVrnr, I'l'R'I'. The value of this parameter is negotiated 
amongst, all nodes on the net, work and is essentially the smallest value selected by 
any node. This defines the maximum amount of time between access to the network 
and should provide for synchronous traffic. 

Although it. can be shown that the algorithm will guarantee that the token will 
return within the negotiated time frame on the average[8], it can not be guaranteed 
that the node will lx: able to hold the token at all once it returns. This has serious 
implications for periodic traflir and maximum throughput and will be examined in 
the next section. 


3 Impact of Token Rotation Time 

Johnson[8] provides analysis concerning the timing requirements of FDDI for both 
the ideal and the non-idcal(including overhead) cases. For the ideal case, the token 
can always be guaranteed to return to the station within 2 + TO PR where TO PR. is 
the current operating value of the TTR.1\ within which the token should return. For 
example, if the currently negotiated value of TOPIt is 125 /tseconds, then the token 
can only be guaranteed to return within 250 //.seconds. It would appear then that 
one would simply negotiate for one-half of the desired TTRT and then the proper 
availability of the token could be assured. For reasons of maximizing utilization of 
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Figure 2: To km Kin/; Protocol Without Kmioval ;il. Destination 

the network, there is compelling motivation l.o have a large value of 7 77/7’ tints 
resulting in a tradeoff brhvrrn 1.1 m? design objective l.o support synchronous traffic, 
and the nerd for high utilization. 'The following section illustrates the impact of 
TTllT on utilization. 


3.1 Parameters affecting TTRT 

As cited in [8], the primary components of ring overhead are as follows. 

• Total Propagation Delay (l) P m V ) is determined l>v multiplying the propagation 
delay for fiber optic media (508- r ) ns/km) hy the length of the network. 

• Latency (/,„) occurs at each node and is effectively the delay between the time 
a hit arrives as a node and departs the node. Therefore, if one examines the 
round trip of a single bit around the network, the delay is increased by the 
latency at each node times the number of nodes. f, toi represents the total 
latency of the notwork( = /V x L ri ). 

• The number of nodes to capture the token, increases the delay of the to- 
ken rotation. In the minimal delay case, no node needs the token and this 
component is nothing, hut no information is transmitted. Dy focusing upon 
the process of transmitting a frame at a single node, this overhead becomes 
apparent. The head of the token arrives at the node and is passed on to the 
rest of the network while the node waiting to transmit identifies this as the 
token. Recognition takes place before the tail of the token is retransmitted, 
providing the capturing node the opportunity to modify the end of the token, 



transforming il, into a non-token frame, and thereby rapturing the token. 1 lie 
delay required to accomplish this is incor |>orat rd into Latency, A„. I lie node 
then pro< reds to transmit its pa< k»*t and re ( ran swil the token to its neighbor- 
ing node as explained in the seel. ion Basic Token Uing and figure 2. 

• As each node captures the token and retransmits it, an additional delay equal 
to the Token Transmission Time (7)) will he incurred. Note that the delay 
from message transmission does not contribute to overhead delay. 

• The design specifications of 1*1)1)1(20] allow for a. maximum J ransmitter Idle 
Time(7’ t ). This represents the time which is required between recognition of 
the token by the node and beginning of transmission of the frame. As in 
the previous item, this is only a factor when nodes are actually capturing the 
token. 

Specified times for these delay components can be found in [20] . 

La.tmcv per connection 000 ns 

'Token 'Transmission Time ns 

Max 'Transmitter Idle Time .'$500 ns 


Consider three scenarios for I* 1)1)1 as a basis for evaluating the impar t of these 
parameters: 

1. 10 nodes separated l»v a * listance ol 1 00 meters each ( 1 Ivin lota I) - representing 
a backbone for interconnecting local area networks, 

2. 3 nodes separated by a distance of 10 meters each (30 meters total) - represent- 
ing the connection of two mainframe/superromputers or peripheral equipment 
which is in a close physical proximity, 

.'$. 500 nodes each separated by a distance* of 100 meters each (50 Kin total) - 
representing a IISLDN or MAN. 

Table I illustrates the delays 2 inherent in each of the scenarios. I lie difference 
between MAX and MIN TOTAL DRLAY is the number of nodes transmitting on a 
token rotation. 

One can see that as latency improves below f>()ns, the effect will be significant 
only in the MIN TOTAL DKLAY for mainframe environments. 

2 a one hit delay is equivalent to 10 ns for a lOOMhps network 
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Latency : GOO ns per node 



1 .Backbone 

2. Main frames 

3.I4SLDN 

Pi op Delay [IDpmp') 

5. 085/, s 

0.152G/as 

2543/as 

Latency (L u> ,) 

G/as 

1 .S/as 

300/ is 

Max Token Trans (71) 

8.S//s 

2. 64 /as 

440/ as 

Max Trans Idle (7’,) 

3 5/ as 

10.5/as 

1750 //s 

MAX TOTAL DELAY 

54.SS5//s 

15. 0926/ as 

5033/zs 

MIN TOTAL DELAY 

1 1.0S5//S 

1 .9526/as 

2S43/as 


Latency : GO ns per node 



1 .Backbone 

2. Main frames 

3.IISLDN 

Prop Delay (D,, rnj ,) 

5.085 //s 

0. 15 26/ as 

2 543/ as 

Latency (L<„,) 

.6/ AS 

.18/ /s 

30/as 

Max Token Trans (71) 

8. 8 /as 

2. 04/ /.s 

440/as 

Max Trans Idle (71) 

35//s 

10.5/as 

jj 

MAX TOTAL DELAY 

49.4 85/ as 

15.092G//S 

| 

MIN TOTAL DELAY 

5.G85/AS 

. 3326/as 

2573/as 


Table 1: Overhead Delay Para meters for KDDI Scenarios 

















3.2 TTRT vs Utilization 


The purpose of this nun lysis is l.o develop n tool which is reasonably simple (.o use 
and which will he able to predict maximum utilization of an KDDI network. Prac- 
tically all aspects of the derivation use average values of the random variables with 
focus given to heavily loaded conditions. Only asynchronous (.radio is considered 
with justification for ignoring synchronous traffic in the analysis being given in the 
following section. The end of the analysis will provide results from a simulation of 
KDDI to illustrate the degree of accuracy of these approximations. 

All nodes on a KDDI network use the same value of II'RI'. If a node does 
not obtain the token in time to transmit its data and maintain the required liming 
restraints, if. simplv forwards the token l.o the next node. One way of viewing 
utilization (U) is to represented it as 


TTRT ~ T RO 
TTRT 


(i) 


where 7* RO represents the Token Potation Overhead. During a single round trip 
of the token, only a certain percentage of the time can be spent sending data. The 
rest of the time essentially represents the amount of time required to transmit the 
token around the ring. Part of T RO is fixed and independent of the network load. 
All other factors remaining fixed, one ran see that for a heavily loaded network, 
increasing TTRT will increase utilization • * at the expense of delay. Here we 
examine an estimation for utilization as a function of TTRT. 

Using the terms developed in the previous sections, I RO ran he expressed as 
follows 

TRO = Nr x (7; i- 7’ ) | / W + Um (2) 

The number of nodes to capture the token on a rotation is dependent upon the 
available bandwidth for transmission (7’77rT - T RO), the packet length, load and 
the number of packets transmitted by each node which raptures the token. All of 
the variables except N r are static values, however, for this derivation, separate the 
components of TRO into two terms as follows, TRO* representing the component of 
TRO which is independent of the number of packets transmitted and the dynamic 
part TRO,. 

TRO = TRO , -(• TRO, (3) 

TRO , = D r ro P + ''tot ('0 

TRO, — N c x (7/ T Ti) (5) 

The dynamic component is determined by the number of nodes which capture 
the token. Each time the token is captured, there is a transmitter idle delay and a 
retransmission of the token. It is possible that a node can transmit multiple packets 
for a single token capture. If each node capturing the token transmits twice as many 
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messages, Uir token rel.ransmission(7’ / ) and transmitter idle delays(7i) would occur 
half ns often. 

Consider the range of values which /V r has with the load uniformly distributed 
among the nodes. Under low loads, few nodes have messages to transmit. As net- 
work load increases, yV r increases, approaching the number of nodes on the network 
A r , then it decreases as the network becomes overloaded. In the last case, as nodes 
have large queues, one token capture results in a large number of packet transmis- 
sions. ICventually, each node holds the token for a period of time which precludes 
other nodes on the network from capturing the token until its next rotation. 

As the queues at each node overload, the utilization actually increases as there 
are fewer token captures per rotation; however, we are interested in determing the 
maximum trallic which the network can support without queue buildup. In such 
an overloaded situation, the network cannot support I he traffic levels even though 
overall utilization may be higher. Therefore, it is assumed that traffic is distributed 
such that on the average a node only has a single packct(or less) to transmit per 
token rotation and that the maximum value of /V,. is /V. The number of packets 
which can be transmitted is dependent upon I, lie number of packets which can be 
transmitted during 1'T 1\T — 77iY)_,. N r would be defined as 


A'w - - 


TTItr - Tito, 

"T TrrrrT 


( 6 ) 


r is the packet length 

If is the transmission rate of the network and 
and i\ f pl is the number of packets transmitted. 


As the load increases, the number <»f nodes rapturing the token would have a 
limit of 

/V,. = N (7) 

and the number of packets transmitted with maximum l.okcn captures N rl m would 
be 

rmr - mo, - n x (7; + t,) 




i^rt Af = 


r 

n 


Substituting Equation 8 back in to Equation 1 , U can be expressed as 

rmr - mo, -i v x (7; -f 7;) 


u = 


rmr 


( 9 ) 


Figures 3 and '1 illustrate I, lie predicted and real maximum utilization for the 
backbone and MAN scenarios listed above. 
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3.3 TTRT Impact, on Synchronous Traffic 

Synchronous traffic has not hoc n included in l.hc previous analysis. Because syn- 
ohronous traffic hy its ual.urc places a uniform load on l.hc system per token rotation, 
we can initially consider if as an overhead of the token rotation. After calculating 
l.hc maximum utilization as described above, add the percentage of synchronous 
traffic to the previous utilization value to obtain the true utilization. The only ap- 
proximation involved is due to the number of token captures which could be higher 
if, for example, synchronous traffic packets were small and distributed to a large 
number of /lodes. This would further reduce maximum utilization. 

Application of the previous results to typical synchronous traffic requirements 
indicates that I'DDI does not support synchronous traffic without significant de- 
crease in utilization. ISDN comparability requires that synchronous traffic must be 
delivered at the rate of once every 125/<s. In or^Icr to guarantee this arrival rate, 
a rTRl of 62.5/xs must be established, for a conservative estimate, assume that 
J T III is I25//.S, knowing that on the average 125//S will be attainable although in 
some instances packets may be lost due to the inability to guarantee TTRT can be 
met. Comparing this with the lower node latency and ignoring packet overhead, the 
maximum utilization is illustrated in Table 2. 

The table indicates that FDD! could not support an acceptable ISDN interface 
in most configurations and that it would be extremely unlikely that synchronous 
traffic with comparable periodicity could bo supported in a long distance (MAN) 
environment. It is also interesting to note that in the scenario for a backbone, the 
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MAN Scenario 




1. Backbone 2. Mainframes 3.HSLDN 

MAX TOTAL DELAY 

60% 88% **** 

MIN TOTAL DELAY 

95.5% 99.9% **** 


Table 2: Maximum Utilization for I 2 r >/rs TTUT 


utilization could drop significantly depending upon the number of nodes which can 
capture the token on one rotation. Phis is of course also dependent on the packet 
size being transmitted. 






4 Scaling FDDI to Gigabit Speeds 

Wil.lt l.lif! d ispa lily I m * I. \v< *( * 1 1 tin* (Irvrlopinrnl. of I In* sprrd of l.ransi Mission technology 
( o | > b i < n I s>si,('ins) • 1 1 1 r I tin* j >« *« m I of pi o« ■« vising rlrmmts in a rompnlrr, one might 
reasonably question the degree to which gigabit networks ran be used at all. This 
disparity between the speed of transmitter technology and processing elements is 
tolerable if large numbers of node's ran be viewed as using the gigabit speeds if 
only for short durations. This is one reason to examine the scalability of FDDI in 
tenns of the number of nodes it will support. [ he previous section also provided 
some insight as to the impact which number of node's will have on performance. The 
second most impoitant factor to examine is network IcufjfJi. Token ring networks arc 
usually dependent upon short propagation delays for providing fast network access. 

Given that this is not a. detailed investigation, the two factors will not bo a.d - 
dressed independently. Instead, the network configurations examined will look at 
increasing the number of nodes while maintaining the same internode distance, 
thereby increasing tin* network length simultaneously. 

When considering the scalability of I* DDI, one could view a. scaling of the trans- 
mitter with proportional scaling of the speed of the nodes, or the transmitter speed 
could be scaled leaving the node processing speeds (values of Transmitter bile Time, 
etc. discussed in the previous section) at the same rate. The data which follows is 
an examination of the latter given its greater probability of occurrence. 

1 he parameter space examined here varies t he number of nodes between 100 and 
.>00 maintaining inl.emode distance at 100m, resulting in network lengths between 
10 Km and 50 Km. 1 he / / /i \ I has been chosen at a level wliic.li allows for maximum 
utilization in the 75-00% range in gencral( 1 0000 /is). The delay terms mentioned in 
the previous section were constant in all runs. Packet size was fixed at 1000 bits. 

Figure 5 shows access delay for slandard( 1 00Mbps) and gigabit FDDI. The ver- 
tical axis scaling has been chosen to be twice 7'7VTF( 1 0000 /is). This shows how 
performance suffers dramatically in long-distance networks when more than one to- 
ken rotation is required to deliver data. As expected, performance degrades in each 
graph as nodes and distance arc increased. The two graphs reveal that gigabit FDDI 
begins to degrade at approximately 70%, whereas standard FDDI degrades slightly 
after 70%. One might expect that a faster transmission rale would mean shorter 
delays, however, these graphs tend to show comparable results. The reason is that 
with a constant packet size there arc more packets at 70% load in gigabit FDDI, 
and more nodes arc also transmitting. The previous section indicated the overhead 
associated with token capture and how it relates to the number of nodes. 

The final graph in figure 5 combines the 100 and 500 nodes curves from the 
previous two graphs. Surprisingly, it does not appear that the scaling of the trans- 
mitter is the issue. Both speeds indicate similar performance curves. Number of 
nodes and distance arc the factors which distinguish the shapes of the curves. 
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5 Performance Improvement 

As l.lu* data transmission rales and distance invert'd by (lie networks increase, Idle 
number of simultaneous packets on a network increases drama tically. The parameter 
a where 

length of iliil.ti ]>al h 

(l = -7 77 7 7 » ( l0 ) 

length oj packet 

represents this concept and frequently arises as a crucial parameter in network per- 
formance. For example, a network of 10 Mbps capacity, 2000 packet length and 
LAN length of I km can only hold .000120 packets at a time. Network designs with 
capacities such as I Cl bps, 100 km and packet lengths of 2000 bytes, could contain 
approximately ill packets simultaneously. 

Increasing the number of packets on a network as shown above demands that 
greater attention be given to the management of these packets. Most LANs do 
not have this problem of simultaneous packets. Networks such as DQDB[17] which 
employs a. slotted scheme or DSM A/RN(4] which uses a hybrid CSMA/CD technique 
may provide greater opportunity for optimization of this packet capacity. 

In addition, the metrics for network performance, or at least our view of them, 
needs to bo reconsidered. When analyzing the performance of a data communica- 
tions network, one typically uses utilization as a metric of evaluation. If a 100Mbps 
network is capable of delivering 100Mbps of data, then the network is assumed to 
have 100% utilization (ignoring packet overhead). 

This 100% limitation is based upon the assumption that only one node is trans- 
mitting at a time. If it is only possible for one node to transmit at a time as in most 
CSMA/CD networks or token ring networks, then this is a reasonable assumption, 
liven in situations where link level protocols provide for the existence of multiple 
packets on the network simultaneously as in selective retransmission and go-back-n , 
and in systems which allow for the building of a train of packets such as Exprcssnot, 
no consideration is given to the possibility' of simnltanenous transmission by nodes. 
Register insertion rings(fi), DSM A/RN(4 J and otbe.rs(l) have shown that utilization 
greater than l00%(Lhroughput greater than 1.0) is achievable. 

Assume that nodes arc numbered from 1,. ,.,/V and placed in a ring. If node 1 
can send to node 2 while node 3 sends to node 4, 200Mbps is being transmitted. To 
understand the inefficiency of a ring running at close to 100% utilization, consider 
not only whether the network is filled with packets, but whether or not the packets 
arc doing useful work, where work which is not useful occurs when packets take up 
network capacity but have already been delivered to the receiver. 

The focus of the suggestion for improvement in performance of FDDI in this 
paper is on recovering the unused packet capacity by removing the packet at the 
destination and inserting new packets in their place which I will call destination 
insertion. Destination insertion will also allow for multiple simultaneous transmit- 
ters on the network and increased throughput. The technique is also applicable to 
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Figure 0: Kcmoving Pac ket at Destination 
DQDB and token rings in gnu'rnl, although the analysis dors not address DQDB. 

5.1 Destination Insertion 

Figure: 2 illustrates the removal of the message at the receiver. In the second part 
of this diagram, MSG - 2 has reached the destination node and is about to arrive 
hack to the original sender, node <1. It has boon shaded to emphasize that the slot 
is only delivering an acknowledgment and that the capacity of the network is not 
being used in an optimal fashion. In this case the slot could have been used by node 
2 to transmit the message to node 3. I.c., two messages could have been delivered 
instead on one with this packet-slot. 

Figure G depicts node 2 removing the message from node 4 and at the same time 
inserting its message to node 3. The removal of the token at node 1 and transmission 
of the message from node l to node 5 is unaffected by this squeezing of the message 
from node 2 to node 3 into the train of packets. The squeezed data can not be 
longer than the message which it is replacing. 

Destination insertion works as follows. When a node transmits a message on the 
network, the message proceeds until it reaches its destination. At that point it is 
marked as received and the slot containing the message is available for further use. 
As long as the slot is on the ring, if can be used by other nodes. This raises two 
additional questions: 



• limv long will the slol. remain on the ring, and 

• which nodes( messages) ;ur c ;mdida ter; for reusing f lic slol.. 

5.2 Restrictions on slot reuse 

This message would normally travel at least as far as the original sender and could 
conceivably travel even further if the original sender release's I, ho token before the 
packet returns. If the time of transmission of the message is at least as long as the 
propagation delay of the network(a < I), the message will terminate at the sender. 
If the time of transmission is much less that the propagation dclay(a > 1), the token 
will leave the node before the message has made a loop and the message may not 
reach the node holding the token for a significant period of time. 

When a < 1, the packet arrives at the sender before transmission is complete. 
Assume that node I holds the token and has a, message for a node t. When node i 
receives the slot, it is reusable by any node up to node I, but not beyond node 1. If 
an attempt is made to send a message past node I, it will be absorbed by node 1, 
the holder of the token. If a > I , additional reuse could be made of the packet, but 
the opportunity for reuse will be terminated at the original sender in this analysis. 

Length of packet also presents a problem, as mentioned above. Obviously, a 
message which is inserted into the free slot must of length less than the length of 
the delivered message. For this analysis, the assumption is made that all packets 
are of equal length with the effect of varying packet lengths to be examined in 
subsequent research. 

5.3 Objective 

If destination insertion recaptures these packets, one expects throughput to increase 
and delay to decrease. The central questions are how much these measures would he 
affected and how feasible the implementation would be. Recent papors[G,l ,4] have 
used similar techniques to show increases on the order of 1.5 to 2 times 100%, but 
these papers have not included any generalizations as to how this might apply in 
an arbitrary case, rather the simulation results are for specific cases. I intend to 
provide an analysis which will allow one to predict the degree to which a method 
such as this can improve performance in KDDI or token rings, and to show that 
a feasible strategy can be developed which does not meet the maximum, but can 
approach it. 

5.4 Advantages 

One might assume that the effect of destination insertion would be simply to provide 
for an increase in throughput at high loads and have little effect at low loads. 
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However, the method enn be shown to improve' performance in the following areas. 

• Throughput will he aide to sustain tradie at a nuieli higher load 

• One of the major problems with l.okon ling networks is the access delay for 
obtaining the token. These extra, slots will reduce average a recss delay to the 
network. 

• Because of the large distance and number of nodes inherent in proposed wide 
area research nets, normal token ring access delays are likely prohibitive. These 
additional packets will reduce access delay as mentioned above and reduce the 
sensitivity of a token ring network such as KDDI to longer distances. 

5.5 Expected Maximum Throughput Increase 

For this analysis, the assumptions will be that all nodes always have at least one 
packet in the queue, that destination address space is uniformly distributed among 
the nodes, and that packets arc' (ixed length. If all messages were destined for the 
neighbor, throughput could be increased by a factor of /V, but this is an unlikely 
scenario. The result derived is a function of the number of nodes, £(«)> which 
slates the factor l>y which throughput enn be expected to improve under heavily 
loaded conditions. For example, if utilization is currently 80% and the expected 
throughput increase, S(n), is I .'I, then utilization should be able to reach 112% 
under the conditions specified in the assumptions above for u nodes. 

In order to determine the expected throughput for such a network, consider the 
traversal of a packet around the network. Assume that node n removes the token 
from the network and transmits a packet. The packet must be destined for one of 
1 1 ic nodes n — 1 ... 1. Assume that the packet is destined for node j,n < j < 1. 
Upon receiving the message, either 

1. node j lias a packet available for transmission to node k where j < k << 1 or 
k = n which states that the message can lie squeezed into the now available 
slot and removed before if passes the original sender 

2. node j has a packet available for transmission to node k where n — 1 < k < j 
and it can not be squeezed without a possibility of being removed by a node 
which has the token (specifically node « may still be transmitting, so we 
assume that it is in order to guarantee viability of flic slot) 

or 

3. there is no packet in the queue (which this analysis is ignoring). 
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Define £(i) lo be I, lie expected increase in thoiighput given that the slot has as 
its destination node /. Using a recursive derivation, start with <f(i). 

••'(') - 7 v CD 

because the expected value of increased throughput is the probability of the message 
in the head of the queue being destined for node n (the original sender of the slot 
and the only possible node to which node 1 can send), jj, times l (the number of 
messages it can send in the slot) plus the probability that the message at the head 
of idle queue is for some other node, times 0. 

The expected increase at the second node is composed of three terms 

1. the probability that the message at the head of its queue is for node n times 
its expected increase, jj times I as above, 

plus 

2. the probability that the message at the head of the queue is for node I Limes 
the expected increase which is I plus £(1) because the message is delivered at 
node I and can be reused again at node ! with expected increase £(l) 

plus 

3. the probability that the message at the head of the node is for neither node 

1 nor ii , ^ , times the expected increase which is also £(I) because the slot 

will proceed to node 1 available for reuse 

Therefore, 


w = 77 + 


’(i) 


A' - 2 
-I — x 




/V A' 

For arbitrary nod*’; j, the formula ran l>r generalized to 


yV - l 

N 7 


c 

C 
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for u < j < 3 
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The first, term in liquation 1 3 represents the expected increase if the message is 
for node 7 /., the original sender. 'The second term represents the expected increase 
if the message is for a node positioned between the current node j and node 1 
which is the squeezed message itself plus any expected increase once that message 
is delivered. The third term assumes that the slot could not he used so it is passed 
on to node j — I. In the ease for j ~ //, the first term is omitted because it would 
never send a message to node o, itself. 

5.6 Overall effectiveness 

The following graph show the increase in throughput expected from a traffic place- 
ment strategy as described above. 'I'lie result of interest from the above derivation 
is the value of £(n.) which describes the number of expected messages delivered 
with each packet as it is transmitted from the node holding the token (n). Figure 
7 shows £(u) versus 11 . One ran observe that the effect of such a technique has a 
much greater cfTect as the number of nodes increases. 

Recall that the proper interpretation of this graph is that throughput can be 
increased by the factor given. Results show that for a 100 node problem which 
operates at 90% maximum utilization, this method will increase throughput by a 
factor of 3 to 270%. A number of curves are provided. The curve marked Analysis 
is the result of calculating £( 77 ) for various values of n as derived above. The 
analytical results can hr* compared with simulation results in the curve Simulation. 
The simulation model did not require using an FDD I model and was only modelling 
the passing of messages from node to node without delay statistics. Here a packet 
was allowed to be reused an arbitrary number of times, an impractical assumption 
discussed later. The final two curves in this figure show maximum utilization when 
one limits the number of times which a slot may be used during a cycle around the 
network. Maj~2 indieal.es that, t.hr* slot, may be used twice (reused once). 


5.7 Additional Simulation Results 

A simulation model of FDDI, written in Simscript, has been developed on Sun work- 
stations in order to Lest performance issues. A modification was made to the model 
to allow for the incorporation of destination insertion. It should be noted that the 
full advantage of this technique can not be seen in these results because of an incon- 
sistency in the original design of the model and the design necessary for destination 
insertion. A new model which will be used to show Lite full potential impact for 
FDDI is under development. The results shown arc a conservative estimate of the 
effect. 

A number of benefits arise from removal of the message at destination and reuse 
of the slot, the first two of which arc investigated in this paper. 
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Figure 7: Kxperlrd Tli rough put Increase 

1. As shown previously, throughput can ho dramatically increased. 

2. I’/vrn in srennrios where the system is not fully loaded, delay characteristics 
arc improved. 

2. The extensibility of an FDDI network is increased. One of the major limita- 
tions of extending an FDDI ring is the large propagation delays experienced 
as total network distance increases. These extra slots will provide additional 
opportunities for transmission beyond the token arrivals, decreasing access 
delay. 

'I. If synchronous traffic is required, if is possible to set 77 *It.T values at a higher 
value and still maintain the same average access delay. Raising the TTRT 
will allow for higher utilization as shown previously. 

The effect of access delay where load is less than 100% is shown in Figures 8 
and 9. TTJIT is set to 5 ins, network ra!.e(/?) at 100 Mbps, packet length^) is 
fixed at 5000 bits. Figure 8 shows how removal airccts the 10 node ease. Figure 
9 incorporates the removal vs non-removal for 10, 50 and 100 nodes. Note that in 
every instance using the destination insertion technique, access delay at 100% load 
is comparable to access delay at very low loads. It should also be noted that these 
runs have not reached the assumption made in the analysis that all nodes have data 
in the queue; however, the effect is still significant. 
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Figure H): Packet Structure 

5.8 Implementation 

The implementation of 1.1 1 is nit pit; is not. without .1 cost., bill. I, lie cost is a nominal 
one. First, the receiver must be able to recognize the destination address and 
mark the packet as delivered. Second, the marking of the packet must be in such 
a manner as to allow for other nodes to determine that the packet has already 
reached the destination. Placing the header at the beginning of the packet allows 
for interpretation of the source and destination. This would make it possible for a 
receiver to determine if the packet had already been delivered, but the process of 
making the decision would be simplest if the receiver marked a 1 bit slot at the end 
of the header to indicate that the packet had been received. All subsequent nodes 
would recognize this similar to recognizing the token and transmit the data. 

Token rings allow the original message to propagate hack to the sender before 
being removed. This provides and acknowledgement mechanism that the message 
has been delivered ami has circulated around the ring correctly. However, receipt of 
the original message is not required. Many protocols only require an acknowledge- 
ment of some type from the receiver. This can be done much more efficiently that 
using the entire packet slot for acknowledgment. If sufficient room is left at the end 
of the packet, the acknowledgement from the receiver can be accommodated. This 
would require an additional header/ trailer combination for the reuse of the packet as 
shown in Figure 10. The number of these extra hcadcr/trailcr combinations would 
be equal to the number of times of expected reuse of the packet. Of course, in the 
best ease, all nodes would want to use the slot requiring N x (Pt Tn iier + Pheader) over- 
head, an unacceptable price to pay. However, a value closer to the average number 
of uses of the slot could he used. For example, if on the average for a given number 
of nodes, the packet would be used 3 times, the overhead of 3 packet/ trailer entries 
would be a reasonable one. 
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G Conclusions 


This paper has shown some of the limitations of KDDI. Those limitations are 
strongly a function of number of nodes and network distance. A tradeoff exists 
between maximum utilization and the network access as determined by the TTRT. 
Return of the token within an average specified time can be guaranteed but use of 
the token cannot. It has been shown that setting TTRT lower reduces utilization. 
Reduced utilization in turn will increase access delay, the reverse of the desired 
effect. 

Scalability of KDDI in the 500 node, 50Km range up to gigabit speeds is not 
without some proportional loss in performance. Simply increasing the transmitter 
speed by a factor of ten does not translate directly into being able to deliver ten 
times the data with similar access delays. 

One method of improving the extensibility of FDDI is to remove packets at the 
destination and make those packets available for reuse as they continue circulating 
around the ring. This has been shown to have an expected increase of over twice 
the capacity of the network for more than 20 nodes when the network is fully loaded 
and to reduce access delay in cases where the network is less than fully loaded. 
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